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DETAILED ACTION 

Election/Restrictions 

Applicants' election without traverse of Group II, Claims 23 to 51 , in the reply filed 
on 13 June 2008 is acknowledged. 

Claims 12 to 22 and 52 to 56 are withdrawn from further consideration pursuant 
to 37 CFR 1 .142(b) as being drawn to a nonelected invention, there being no allowable 
generic or linking claim. Election was made without traverse in the reply filed on 13 
June 2008. 

Claims 1 to 1 1 are generic. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 1 to 1 1 and 23 to 51 are rejected under 35 U.S.C. 112, first paragraph, as 
failing to comply with the written description requirement. The claims contain subject 
matter which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventors, at the time the application 
was filed, had possession of the claimed invention. 

Independent claims 1 and 23 are amended to contain the term "modality 
dependent", which is new matter because Applicants' Specification as originally-filed 
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does not provide an adequate written description in such a way as to reasonably convey 
that the inventors had possession of the concept of modality dependence. The 
Specification does not set forth the term "modality dependent", and the only disclosure 
of the term is from Dantzig et al., the prior art from which Applicants are attempting to 
distinguish. 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 1 to 11 and 23 to 51 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

The limitations of independent claims 1 and 23 that attributes are "directly 
related" to visual rendering, and modality dependent attributes being provided "directly" 
from the controls on the authoring page, are vague and indefinite. It is appreciated that 
Applicants are attempting to distinguish by inclusion of the term "directly related" over 
the modality independent script of Dantzig et al. However, Applicants' Specification 
does not expressly disclose anything about the attributes being "directly related" to the 
visual rendering or using modality dependent attributes provided "directly" from the 
controls on an authoring page, nor would it be immediately clear to one having ordinary 
skill in the art that the attributes are "directly related" in any sense not necessarily 
disclosed by Dantzig et al. The limitation "directly related" is vague, and may be 
misdescriptive. 
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Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1 to 2 and 4 to 8 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Dantzig et al. 

Regarding independent claim 1, Dantzig etal. discloses a system and method for 
generating multi-modal applications from markup scripts, comprising: 

"a set of modality dependent controls defined in an authoring page for a website 
for defining desired visual renderings and at least one of recognition and audible 
prompting on a client in a client/server system, each control having a first set of 
attributes directed related to visual rendering and a second set of attributes directed 
related to at least one of recognition and audibly prompting, the controls being related to 
client side markup executable by a client browser" - an XML (extensible Markup 
Language) script is implemented in a single authoring format ("an authoring page") 
(column 5, lines 50 to 56); main Tenderer 14 of a multi-modal presentation manager 1 1 
initiates a first processing thread comprising a GUI presentation manager 15 ("a first set 
of attributes directly related to visual rendering") (column 7, lines 38 to 43: Figure 1); 
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presentation of a graphic user interface (GUI) for an application defines a "desired 
visual rendering"; main renderer 14 of a multi-modal presentation manager 11 initiates a 
second processing thread comprising a speech renderer 16 ("a second set of attributes 
directly related to at least one of recognition and audibly prompting"), wherein the 
speech renderer 16 utilizes a plurality of speech engines 17 for speech recognition and 
text-to-speech synthesis (column 7, lines 38 to 47: Figure 1); controls are "modality 
dependent" because each processing thread is directed to either a modality relating to 
GUI presentation or a modality relating to a speech renderer; multi-modal presentation 
manager 1 1 controls an application on a web browser or a desktop (column 8, lines 32 
to 35: Figure 1); one thread comprising a GUI presentation manager 15 is "directly 
related" to defining desired visual renderings on the client device because the thread 
immediately initiates a visual modality; similarly, a second thread comprising a speech 
renderer 16 is "directly related" to defining desired operation on the client device 
because the thread immediately initiates speech recognition or text-to-speech 
synthesis; 

"a module operable on a computer, the module being configured to receive the 
authoring page, and wherein the module is further configured to generate, using 
modality dependent attributes provided directly from controls on the authoring page, 
client side markup executable by the client browser on the client in the server/client 
system in accordance with the controls and the attributes of the controls to perform both 
visual rendering and at least one of recognition and audibly prompting" - multi-modal 
presentation manager 1 1 controls an application on a web browser or a desktop 
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(column 8, lines 32 to 35: Figure 1); implicitly, a web browser is executed on a client in a 
client/server architecture for receiving information from the Internet; a "single-authoring" 
system and method is an interaction-based programming paradigm for creating content 
as an intent-based markup script (column 5, line 20 to column 6, line 2; column 10, lines 
24 to 28); thus, authoring for web-based presentation is on "an authoring page" at a 
client browser; main renderer 14 of a multi-modal presentation manager 11 initiates a 
first processing thread comprising a GUI presentation manager 15 ("a first set of 
attributes directly related to visual rendering") (column 7, lines 38 to 43: Figure 1); 
presentation of a graphic user interface (GUI) for an application defines a "desired 
visual rendering"; main renderer 14 of a multi-modal presentation manager 11 initiates a 
second processing thread comprising a speech renderer 16 ("a second set of attributes 
directly related to at least one of recognition and audibly prompting"), wherein the 
speech renderer 16 utilizes a plurality of speech engines 17 for speech recognition and 
text-to-speech synthesis (column 7, lines 38 to 47: Figure 1). 

Regarding claims 2 and 4, Dantzig etal. discloses that controls relate to 
grammars for speech recognition (column 9, lines 31 to 39; column 16, lines 6 to 30). 

Regarding claims 5 and 6, Dantzig etal. discloses that controls relate to XML 
(column 5, lines 50 to 56), VoiceXML (a form of XML) (Abstract), and WML (column 6, 
lines 56 to 62). 

Regarding claims 7 and 8, Dantzig etal. discloses a speech renderer 16 
generates audible output by text-to-speech synthesis (column 7, lines 42 to 45). 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 3, 9 to 11, and 23 to 46 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Dantzig et al. in view of Ladd et al. ('336). 

Concerning independent claim 23, Dantzig etal. discloses a system and method 
for generating multi-modal applications from markup scripts, comprising: 

"a first set of modality dependent visual controls having attributes defined on an 
authoring page for a website directly having attributes directly related to defining desired 
visual renderings on the client device, the first set of controls being related to client side 
markup executable by a client browser" - main Tenderer 14 of a multi-modal 
presentation manager 1 1 initiates a first processing thread comprising a GUI 
presentation manager 15; controls are "modality dependent" because each processing 
thread is directed to either a modality relating to GUI presentation or a modality relating 
to a speech renderer ("a first set of modality dependent visual controls having attributes 
. . . related to defining desired visual renderings") (column 7, lines 38 to 43: Figure 1); 
an XML (extensible Markup Language) script is implemented in a single authoring 
format ("defined on an authoring page for a website") (column 5, lines 50 to 56); 
presentation of a graphic user interface (GUI) for an application defines a "desired 
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visual rendering"; multi-modal presentation manager 11 controls an application on a 
web browser or a desktop (column 8, lines 32 to 35: Figure 1 ); implicitly, a web browser 
is executed on a client in a client/server architecture for receiving information from the 
Internet; 

"a second set of modality dependent controls defined on the authoring page for 
operation on the client device having attributes directly related to at least one of 
recognition and audible prompting, ... the second set of controls being selectively 
associated with the first controls, and the second set of controls being related to client 
side markup executable a client browser" - main renderer 14 of a multi-modal 
presentation manager 1 1 initiates a second processing thread comprising a speech 
renderer 16, wherein the speech renderer 16 utilizes a plurality of speech engines 17 for 
speech recognition and text-to-speech synthesis (column 7, lines 38 to 47: Figure 1); 
controls are "modality dependent" because each processing thread is directed to either 
a modality relating to GUI presentation or a modality relating to a speech renderer ("a 
second set of modality dependent controls having attributes directly related to at least 
one of recognition and audible prompting") (column 7, lines 38 to 43: Figure 1); an XML 
(extensible Markup Language) script is implemented in a single authoring format 
("defined on an authoring page for a website") (column 5, lines 50 to 56); multi-modal 
presentation manager 1 1 controls an application on a web browser or a desktop 
(column 8, lines 32 to 35: Figure 1); implicitly, a web browser is executed on a client in a 
client/server architecture for receiving information from the Internet; in deferred 
rendering and presentation, a speech renderer 16 ("a second set of controls") is 
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"selectively associated with" GUI presentation manager 15 ("a first set of controls") 
because multi-modal presentation manager 11 automatically integrates and 
synchronizes voice synthesis and speech recognition functions with the presentation 
layer of applications (column 6, line 63 to column 7, line 8: Figure 1 ); 

"a module operable on a computer, the module being configured to receive the 
authoring page, wherein the module is further configured to process the controls of the 
first set in the authoring page to generate client side markup by incorporating the 
attributes in the modality dependent controls that is executable by the client browser on 
the client in the server/client system in accordance with the controls of the first set and 
the attributes of the controls of the first set to perform both visual rendering, and 
wherein the module is configured to process the controls of the second set to generate 
client side markup by incorporating the attributes in the modality dependent controls that 
is executable by the client browser on the client in the client/server system in 
accordance with the controls of the second set and the attributes of the controls of the 
second set in the authoring page to perform at least one of recognition and audibly 
prompting" - main Tenderer 14 of a multi-modal presentation manager 1 1 initiates a 
second processing thread comprising a speech Tenderer 16, wherein the speech 
renderer 16 utilizes a plurality of speech engines 17 for speech recognition and text-to- 
speech synthesis (column 7, lines 38 to 47: Figure 1); controls are "modality dependent" 
because each processing thread is directed to either a modality relating to GUI 
presentation or a modality relating to a speech Tenderer ("a second set of modality 
dependent controls") (column 7, lines 38 to 43: Figure 1); an XML (extensible Markup 
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Language) script is implemented in a single authoring format ("the authoring page") 
(column 5, lines 50 to 56); authoring produces content for both GUI presentation 
manager 15 and speech renderer 16 (column 7, lines 38 to 48). 

Concerning independent claim 23, Dantzig et al. discloses grammars in 
VoiceXML in order to optimize speech recognition functions (column 10, lines 38 to 56), 
but omits the limitations of "wherein attributes directly related to recognition include at 
least one of location of grammar for use in recognition and confidence level thresholds 
for recognition and wherein attributes directed related to audible prompting include at 
least one of inline text for text-to-speech conversion, location of data for audible 
rendering and playing of a prerecorded audio file". However, Ladd et al. ('336) teaches 
a voice browser for interactive services, where a GRAMMAR input includes a SCR 
attribute that can be a grammar address (i.e., a URL) for a markup language document: 
SCR = "gram//.SomeGrammar/month/year" ("location of a grammar for use in 
recognition"). (Column 20, Line 47 to Column 21 , Line 1 ) Moreover, Ladd et al. ('336) 
provides a voice browser, where a PROMPT element of the markup language is used to 
define content by <PROMPT> text </PROMPT> that is read by a text-to-speech unit, so 
that markup of <PROMPT> Please select a soft drink. </PROMPT> includes at least 
"inline text for text-to-speech conversion". (Column 16, Line 63 to Column 17, Line 21 ; 
Column 18, Lines 33 to 39) An objective is permit users to access information from any 
location in the world via any suitable network access device. (Column 43, Lines 54 to 
63) It would have been obvious to one having ordinary skill in the art to include markup 
attributes relating to a location of a grammar and inline text for text-to-speech 
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conversion as taught by Ladd et al. ('336) in a system and method for generating and 
presenting multi-modal applications from markup scripts of Dantzig et al. for a purpose 
of permitting users to access information from any location in the world via a suitable 
network access device. 

Concerning claims 3 and 25, Ladd etal. ('336) discloses attributes for grammars 
(column 13, lines 6 to 10), and retrieving grammars from database locations (column 
12, lines 7 to 14; column 14, lines 18 to 28) for speech recognition. 

Concerning claims 9 to 1 1 , and 31 to 33, Ladd et al. ('336) discloses determining 
an address for playing a prompt to a user (column 1 3, line 66 to column 1 4, line 1 7: 
Figure 5a: Steps 400, 402, 406); both recorded sound samples (column 15, line 63) and 
text to speech (TTS) (column 16, lines 11 to 20) are provided. 

Concerning claims 24 and 26, Dantzig etal. discloses controls relate to 
grammars for speech recognition (column 9, lines 31 to 39; column 16, lines 6 to 30). 

Concerning claims 27 and 28, Dantzig etal. discloses controls relating to XML 
(column 5, lines 50 to 56), VoiceXML (a form of XML) (Abstract), and WML (column 6, 
lines 56 to 62). 

Concerning claims 29 and 30, Dantzig etal. discloses a speech renderer 16 
generates audible output by text-to-speech synthesis (column 7, lines 42 to 45). 

Concerning claims 34 to 46, Dantzig et al. discloses a system and method for 
generating and presenting multi-modal applications from markup scripts for 
synchronizing a GUI presentation layer with voice synthesis and speech recognition, but 
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omits details relating to "attributes providing a reference to a location", "a prerecorded 
audio data file", "an identifier of the associated control", "a question control", "an answer 
control", "binding the recognition value", and "a confirmation control". However, Ladd et 
al. ( ! 336) teaches a voice browser for interactive services. An objective is permit users 
to access information from any location in the world via any suitable network access 
device. (Column 43, Lines 54 to 63) It would have been obvious to one having ordinary 
skill in the art to include details disclosed by Ladd et al. ('336) in a system and method 
for generating and presenting multi-modal applications from markup scripts of Dantzig 
et al. for a purpose of permitting users to access information from any location in the 
world via a suitable network access device. 

Concerning claim 34, Ladd et al. ('336) discloses a markup language for text to 
speech; implicitly, when the text is displayed and the speech is produced for an audible 
prompt, there is an association of attributes between visual controls and audible 
controls. 

Concerning claims 35 to 37, Ladd etal. ('336) discloses an option list in a markup 
language for controlling which choices are available at a network access apparatus 
(column 28, lines 9 to 60). 

Concerning claim 38, Ladd et al. ('336) discloses a FORM input to collect an 
order in response to a prompt, and post the input to an address (column 20, lines 20 to 
46); thus, a markup language controls a prompt, then activates an input, and then 
performs a post operation. 
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Concerning claims 39 to 43, Ladd et al. ('336) discloses a markup language for 
generating an audible prompt of a question and a grammar for an answer; an answer is 
followed by, and is activated, a question prompt, where an answer is bound for 
recognition by <INPUT TYPE> (column 18, lines 40 to 55); a post operation is "an event 
related to operation of binding" (column 20, lines 28 to 46). 

Concerning claims 44 to 46, Ladd et al. ('336) discloses a markup language for 
re-prompting ("repeating an audible prompt") (column 14, line 57 to column 15, line 16: 
Figure 5a: Steps 416, 425), and an attribute for confirming a recognition result (column 
15, lines 45 to 54: Figure 5a: Step 452). 

Claims 47 to 51 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dantzig et al. in view of Ladd et al. ('336) as applied to claims 23, 39, 40, 45, and 46 
above, and further in view of WCW Working Draft ("Grammar Representation 
Requirements for Voice Markup Languages"). 

Ladd et al. ('336) discloses a confirmation control to accept an answer as a 
recognized result that is correct (column 15, lines 44 to 59: Figure 5b: Step 456). Lack 
of confirmation implicitly denies a recognized result, whereupon the process continues 
to replay a prompt for a current step so as to correct a recognition result. (Figures 5a 
and 5b: Step 446) However, Ladd et al. ('336) omits an attribute related to a confidence 
level for confirming, accepting or denying, and correcting a recognition result. WCW 
Working Draft teaches grammars for voice markup languages with attributes, where 
confidence scoring tightens or relaxes the normal rejection constraints to provide 
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content based control of performance. (Sections 4.3 and 5.1 ) It would have been 
obvious to one having ordinary skill in the art to provide confidence scoring as taught by 
WCW Working Draft in the voice browser for interactive services of Ladd et al. ('336) for 
a purpose of tightening or relaxing rejection constraints to provide content based control 
of performance. 



Response to Arguments 

Applicants' arguments filed 16 April 2008 have been fully considered but they are 
not persuasive. 

Firstly, Applicants argue that the limitation of "directly related" is not indefinite 
under 35 U.S.C. §112, 2 nd If. Applicants say that controls for visual display such as 
location for rendering, font, foreground color, background color, etc., are expressly 
recited so as to remove any doubt as to what directly related means. 

However, it is respectfully maintained that reciting examples of the controls does 
not make the term "directly related" any more definite. It is the nature of the relationship 
alleged to be "direct" that gives rise to the indefiniteness, not the nature of the controls. 
Again, it is recognized that Applicants are attempting to distinguish over Dantzig et al., 
the prior art, by including the limitation of "directly related" as basis for patentability. 
However, the limitation of "directly related" is itself indefinite, because it is not clear in 
what sense the relationship is "direct", or how a similar relationship in any prior art might 
be correspondingly "indirect". 
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Secondly, Applicants argue that Dantzig et al. does not in any way define the 
term "directly related" so as to cast doubt as to the definiteness of the limitation "directly 
related". Applicants say that they have provided evidence that the Specification 
provides specific attributes that one skilled in the art would understand are "directly 
related". 

Here, it should be pointed out that there is no requirement of showing that the 
prior art renders a limitation indefinite under 35 U.S.C. §112, 2nd 1J. Clearly, an 
indefiniteness rejection does not rely upon prior art. Still, Applicants' position that the 
specific attributes that are "directly related" are disclosed by the Specification is 
disingenuous because Applicants' arguments are predicated on a distinction between 
attributes being modality-independent or modality-dependent so as distinguish over 
Dantzig et al., and not on any specific examples of attributes such as location for 
rendering, font, foreground color, background color, etc. 

Thirdly, Applicants attempt to distinguish over Dantzig et al. by saying that the 
controls are "modality dependent". 

However, Applicants' limitations directed to controls that are "modality 
dependent" present issues of new matter as not being described by Applicants' 
originally-filed Specification. Applicants' Specification does not appear to expressly 
disclose the term "modality dependent", and the only place that term is found is in 
Dantzig et al. It is improper under 35 U.S.C. §112. 1 st % for Applicants to copy a term 
found only in the prior art, and to claim it as their invention, when the term is not 
adequately described by their originally-filed Specification. 
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Moreover, it is maintained that the individual threads are "modality dependent" for 
Dantzig et al. One thread comprises a GUI presentation manager and a second thread 
comprises a speech renderer. (Column 7, Lines 38 to 51) Dantzig etal. expressly 
refers to these as "modality-specific representations". (Column 7, Lines 25 to 30) 

Therefore, the rejections of claims 1 to 11 and 23 to 51 under 35 U.S.C. §112, 1 st 
as failing to comply with the written description requirement; of claims 1 to 1 1 and 23 
to 51 under 35 U.S.C. §112, 2 nd 1f, as being indefinite for failing to particularly point out 
and distinctly claim the subject matter; of claims 1 to 2 and 4 to 8 under 35 
U.S.C. §1 02(e) as being anticipated by Dantzig et al.; of claims 3, 9 to 1 1 , and 23 to 46 
under 35 U.S.C. §1 03(a) as being unpatentable over Dantzig et al. in view of Ladd et al. 
('336); and of claims 47 to 51 under 35 U.S.C. §1 03(a) as being unpatentable over 
Dantzig et al. in view of Ladd et al. ('336), and further in view of WCW Working Draft, 
are proper. 

Conclusion 

Applicants' amendment necessitated the new grounds of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicants are reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
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mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to MARTIN LERNER whose telephone number is 
(571 )272-7608. The examiner can normally be reached on 8:30 AM to 6:00 PM 
Monday to Thursday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David R. Hudspeth can be reached on (571) 272-7843. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
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Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Martin Lerner/ 
Primary Examiner 
Art Unit 2626 
July 30, 2008 



